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^\j Abstract 



1.^ , In this note we study packing or covering integer programs with at most k constraints, which 

j-~r • are also known as k- dimensional knapsack problems. For integer fc > and real e > 0, we observe 

there is a polynomial-sized LP for the fc-dimensional knapsack problem with integrality gap at 
Cn ' most 1 + e. The variables may be unbounded or have arbitrary upper bounds. In the (classical) 

packing case, we can also remove the dependence of the LP on the cost-function, yielding a 

polyhedral approximation of the integer hull. This generalizes a recent result of Bienstock [3] 

on the classical knapsack problem. 

H: 

. 1 Introduction 

CS| ■ The classical knapsack problem is the following: given a collection of items each with a value and 

^ ', a weight, and given a weight limit, find a subset of items whose total weight is at most the weight 

p^ I limit, and whose value is maximized. If n denotes the number of items, this can be formulated as 

cn ' the integer program {max^^^j^ XiVi \ x £ {0, 1}", X^"=i XiWi < £} where n denotes the number of 



items, Vi denotes the value of item z, wi denotes the weight of item i, and £ denotes the weight 



^O ; limit. 

^— s ■ In the more general k-dimensional knapsack (or fc-constrained knapsack) problem, there are k 

different kinds of "weight" and a limit for each kind. An example for A; = 3 would be a robber 
who is separately constrained by the total mass, volume, and noisiness of the items he is choosing 
to steal. An orthogonal generalization is that the robber could take multiple copies of each item i, 
up to some prescribed limit of di available copies. We therefore model the fc-dimensional knapsack 

_C^ I problem as 

{maxCT I a; G Z",0 < X < d,ylx < 6} (1) 

where ^4 is a k-by-n matrix, 6 is a vector of length fc, and d is a vector of length n, all non-negative 
and integral. Two special cases are common: if d = 1 we call it the 0-1 knapsack problem; if 
d = -|-oo, we call it the unbounded knapsack problem. 

Another natural generalization is the k-dimensional knapsack- cover problem, 

{min ex I X G Z", < X < d, ylx > 6} 

which has analogous unbounded and 0-1 special cases. We sometimes call this version the covering 
version and likewise (1) is the packing version. 
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On the positive side, for any fixed k, all above variants admit a simple pseudo-polynomial- 
time dynamic programming solution. Chandra et al. [9] gave the first PTAS (polynomial-time 
approximation scheme) for /c-dimensional knapsack in 1976, and later an LP-based scheme was given 
by Frieze and Clarke [13]. See the book by Kellerer et al. [18, §9.4.2] for a more comprehensive 
literature review. The case k = 1 also admits a fully polynomial-time approximation scheme 
(FPTAS), but for /c > 2 there is no FPTAS unless P=NP. This was originally shown for 0-1 
fc-dimensional knapsack by Gens & Levner [15] and Korte & Schrader [20] (see also [18]) and 
subsequently for arbitrary d by Magazine & Chern [24]. 

Our main result is the following: 

Theorem 1. Let k and e be fixed. Given a k-diniensional knapsack (resp. knapsack-cover) instance 
/C, there is a polynomial- sized extended LP relaxation C of V with OPT('P) > (1 — e)OPT(£) 
(resp. with OPT(P) < (1 + e)OPT(£);. 

Here "polynomial-sized extended LP relaxation" means the following. First, V has n variables. 
Then C must have those n variables plus a polynomial number of other ones. The projection C of 
C onto the first n variables must contain the same integral solutions as V. Finally, C and V must 
have the same objective function, i.e. the objective function should ignore the extended variables. 

In the proof, we will see that the LP can be constructed in polynomial time, and that a near- 
optimal integral solution can be obtained from an optimal extreme point fractional solution just 
by rounding down (resp. up). The number of variables in the LP is n^y^l'^i and the number of 
constraints is kn^ '''> . The integrality gap of an IP is the worst-case ratio between the fractional 
and integral optimum and therefore Theorem 1 can be equivalent stated as saying that V has 
integrality gap at most 1 -|- e. 

Our result and the techniques we use are a generalization of a recent result of Bienstock [3], 
which dealt with the packing version for k = 1. The key observation we contribute is that his 
"filtering" approach was also traditionally used to get a PTAS for multi-dimensional knapsack; in 
filtering we exhaustively guess the 7 max-cost items in the knapsack for some constant 7. 

The construction of C in Theorem 1 turns out to depend on the cost function c. A more 
interesting and challenging problem is to find an C which is independent of the cost-function, since 
this gives a polyhedral approximation C of V e.g. in the packing case, it implies £' D P D (1 — e)£'. 
Bienstock's result [3] actually gives an LP which does not depend on the item cost/profits c. We 
will show (in Section 4) that in the packing case, our approach can be similarly revised: 

Theorem 2. Let k and e he fixed. Given a k-dimensional knapsack instance KL, there is a 
polynomial- sized extended LP relaxation C of V with OPT('P) > (1 — e)0PT(i2), such that C 
does not depend on c. 

This comes as the cost of an increase in size to kn^^^ ''^' . For the covering case performing the 
same (a polynomial-sized extended LP relaxation independent of c with integrality gap < 1 + e) is 
an interesting open problem; we elaborate at the end. 

1.1 Related Work 

Knapsack (whether packing or covering) has an FPTAS by dynamic programming, and it is well- 
known that dynamic programs of such a form can be solved as a shortest-path problem, which has 
an LP formulation. Nonetheless, there is no evident way to combine these steps to get an LP for 



knapsack with integrality gap 1 + e. The problem (say, for packing, which is simpler) is that last 
step in the FPTAS is not merely to return the last entry of the DP table, but rather it finds the 
maximum scaled profit such that the minimum volume to obtain it fits inside the knapsack (and 
then recovers the actual solution). The naive fix is adding this volume constraint to the LP but it 
makes the LP non-integral and then it is not clear how to proceed. 

Bienstock & McClosky [4] extend the work of Bienstock [3] to covering problems and other 
settings, and also give an LP of size n^(l/e) ^ ^°^ ^ with integrality gap 1 + e for 1-dimensional, 0-1 
covering knapsack.^ There is some current work [5] on obtaining primal-dual algorithms (that is, 
not needing the ellipsoid method or interior-point subroutines) for knapsack-type covering problems 
with good approximation ratio and [4] reports that the methods of [5] extend to a combinatorial 
LP-based approximation scheme for 1-dimensional covering knapsack. 

Answering an open question of Bienstock [4] about the efficacy of automatic relaxations for the 
knapsack problem, Karlin et al. [17] recently found that the "Laserre hierarchy" of semidefinite 
programming relaxations, when applied to the 1-dimensional 0-1 packing knapsack problem, gives 
an SDP with integrality gap 1 + e after 0(l/e^) rounds. 

Knapsack problems have a couple of interesting basic properties. The first contrasts with 
Lenstra's result [22] that for any fixed k, integer programs with k constraints can be solved in poly- 
nomial time; in comparison, if we have nonnegativity constraints for every variable plus one other 
constraint, we get the unbounded (1-dimensional) knapsack problem, which is NP-hard [23]. Sec- 
ond, recall that for any optimization problem whose objective is integral, and whose optimal value 
is polynomial in the input size, any FPTAS can be used to get a pseudopolynomial-time algorithm. 
In contrast, 0-1 2-dimensional knapsack shows the converse is false: it has a pseudopolynomial-time 
algorithm, but getting an FPTAS is NP-hard even when each profit Cj is 1, e.g. see [18, Thm. 9.4.1]. 

There is a line of work on maximizing constrained submodular functions. For non- monotone 
submodular maximization subject to k linear packing constraints, the state of the art is by Lee et 
al. [21] who give a (5 -|- e)-approximation algorithm. For monotone submodular maximization the 
state of the art is by Chekuri & Vondrak [10] who give a (e/(e — 1) -|- e)-approximation subject 
to k knapsack constraints and a matroid constraint. We note it is NP-hard to obtain any fac- 
tor better than e/(e — 1) for monotone submodular maximization over a matroid [12], so in this 
setting knapsack constraints only affect the best ratio by e, just like in our setting of LP-relative 
approximation. 

1.2 Overview 

First, we review rounding and filtering. Rounding is a standard approach to turn an optimal 
fractional solution into a nearly-optimal integral one, and here we lose up to k times the maximum 
per-item profit. Filtering works well with rounding because it reduces the maximum per-item 
profit; the power of these ideas is already enough to get an LP-based approximation scheme [13], 
but it uses a separate LP for each "guess" made in filtering. Therefore, like Bienstock [4], we use 
disjunctive programming [2] to combine all the separate LPs into a single one. The approach has 
some similarity to the knapsack-cover inequalities of Carr et al. [6]. 



^They use a disjunctive program; in essence, the LP guesses the most costly item in the knapsack, then for 
1 = 1,..., 0{- log i) it guesses the number of items whose costs are (1 + i)~('''+il times that cost, with all guesses 
> - deemed equivalent. In particular the LP depends on the cost function. We remark that the method does not 
readily extend to fc-dimensional knapsack. 



2 Rounding and Filtering 

We now explain the approach. A knapsack instance (1) is determined by the parameters {A, b, c, d). 
The naive LP relaxation of the knapsack problem is 

{max ex I X G M",0 < X < (i,^x < 6}. IC{A,b,c,d) 

In the following, fractional means non-integral. The following lemma is standard. 

Lemma 3. Let x* be an extreme point solution to the linear program {lC{A,b,c,d)). Then x* is 
fractional in at most k coordinates. 

Proof. It follows from elementary LP theory that x* S M" satisfies n (linearly independent) con- 
straints with equality. There are k constraints of the form AjX < bj; all other constraints are of 
the form Xj > or Xj < di, so at least n — koi them hold with equality. Clearly x j > and Xj < di 
cannot both hold with equality for the same i, so x* G {0,dj} for at least n — k distinct i, which 
gives the result. D 

Therefore, we obtain the following primitive guarantee on a rounding strategy. Let [-J applied 
to a vector mean component- wise floor and let Cmax '■= niaxj Cj. 

Corollary 4. Let x* be an extreme point solution to the linear program {!C{A,b,c,d)). Then 

C \_X J ^ ex rCC^aSiX • 

Now the idea is to take x* to be an optimal fractional solution, and use filtering (exhaustive 
guessing) to turn the additive guarantee into a multiplicative factor of 1 + e. Let 7 denote a 
parameter, which represents the size of a multi-set we will guess. For a non-negative vector z let 
the notation ||2:||i mean ^^ Zj. A guess is an integral vector g with < g < d, Ag < b and \\g\\i < 7. 
It is easy to see the number of possible guesses is bounded by (n + l)'*', and that for any constant 
7 we can iterate through all guesses in polynomial time. 

From now on we assume without loss of generality (by reordering items if necessary) that 
ci < C2 < • • • < c„. For a guess g with \\g\\i = 7 we now define the residual knapsack problem 
for g. The residual problem models how to optimally select the remaining objects under the 
restriction that the 7 most profitable^ items chosen (counting multiplicity) are g. Let fi{g) denote 
min{i | gi > 0}. Define d^ to be the first fi{g) coordinates oi d — g followed by n — iJL{g) zeroes, 
and b^ = b — Ag. The residual knapsack problem for g is {A, b^ , c, d^). The residual problem for g 
does not permit taking items with index more than fi{g) and so its Cmax value may be thought of 
as c^(g) or less, which is at most c • (7/||(7||i = c • 5/7. 

If a guess g has H^Hi < 7, define b^ and d^ to be all-zero. Then Corollary 4 gives the following. 

Corollary 5. Let xqpt be an optimal integral knapsack solution for {A,b,c,d). Let g be the 7 
most profitable items in xqpt (or all, if there are less than j). Let x* be an optimal extreme point 
solution to IC{A,W ,c,d^). Then g + [x*J is a feasible knapsack solution for {A,b,c,d) with value 
at least 1 — k/^ times optimal. 



^To simplify the description, even if Ci+i — d we think of item i + 1 as more profitable than item i. 



Proof. We use OPT to denote c • xqpt- Note that xqpt — 5 is feasible for the residual problem for 
g. Therefore c ■ x* > OPT — c ■ g. Moreover Cmax in the residual problem for g is not more than 
c • 5/7 < ^^ ) so Corollary 4 shows that 

OPT OPT 

c • I X* I >c-x* - k > OPT -c-g- k 

7 7 

and consequently [x*\ + g is a solution with value at least 0PT(1 — -), as needed. D 

By taking 7 = k/e and solving fC(A,b^ ,c,d^) for all possible g we get the previously known 
PTAS for fc-dimensional knapsack; we now refine the approach to get a single LP. 

3 Disjunctive Programming 

We now review some disjunctive programming tools [2]. The only result we need is that it is possible 
to write a compact LP for the convex hull of the union of several polytopes, provided that we we 
have compact LPs for each one. 

Suppose we have polyhedra P^ = {x G M" | A^x < b^} and P^ = {x G M" [ A'^x < 6^}. Both of 
these sets are convex and it is therefore easy to see that the convex hull of their union is the set 

conv.hull(P^ U P^) = {x gM" I x = Ax^ + (1 - A)x2,0 < A< l,A^x^ <b^,A^x^ < 6^}. 

However, this is not a linear program, e.g. since we multiply the variable A by the variables x^. 
Nonetheless, it is not hard to see that the following is a linear formulation of the same set: 

conv.hull(P^ U P2) = {x G M" I X = x^ + x^, < A < 1, A^x^ < Xb^,A^x^ < (1 - >^)b^}- 

A similar construction gives the convex hull of the union of any number of polyhedra; we now apply 
this to the knapsack setting. 

The LP IC{A, b^ ,c, d^) was constructed to mean the left-over problem after making a guess g of 
the 7 most profitable items; we similarly shift this LP to get {y = x + g\x^ M", < x < d^, Ax < 
b^} which is the same set, after the guessed part is added back in. 

Let Q denote the set of all possible guesses g. Then the convex hull of the union of the shifted 
polyhedra is given by the feasible region of the following polyhedron: 

|y I y = ^ y9. ^ ^9 = 1- ^ > 0; V5 : y» = x^ + X^g, < x^ < A^d^ Ay3 < X^b^y (£) 

We attach objective maxc • y to {C) to make it into an LP, and use it to prove Theorem 1. 

Proof of Theorem 1, packing version. Let y be an optimal extreme point solution for (£). It is 
straightforward to argue that any extreme point solution has A^ = 1 for some particular g* , and 
A^ = for all other g. Hence y = x^ +9* where x^ is an optimal extreme point solution to 
1C{A,W ,c,d^ ). We now show that [y\ is a (1 — e)-approximately optimal solution, re-using the 
previous arguments. 

If ||5*||i < 7, then x^ = so y is integral, hence y is an optimal knapsack solution. Otherwise, 
if ||g'*||i = 7, then Corollary 4 shows that 

* * c ■ a* c-g* 
c ■ [y\ = c ■ [x^ \ + c ■ g* > c ■ x^ — k \- c ■ g* = c ■ y — k > (1 — e)c • y, 

7 7 

which completes the proof. D 



The corresponding result for the covering version is very similar. One difference is that we 
round up instead of down. The other is that some guesses become inadmissible. Let g be an 
integral vector with < g < d, \\g\\i < 7; we define fi{g),d^ as before and call g a guess only if 
A{g + d^) > b, in which case we set b^ to be the component- wise maximum of and b — Ag. 

4 Removing the Dependence on c for Packing Problems 

In the LPs described above, for each guess g, we treated that guess as the set of most profitable 
items. In particular, b^ and d^ are defined in a way that depends on c. We now show in the 
packing case, how to write a somewhat larger LP, still with integrality gap 1 + e, which is defined 
independently of c. This exactly follows the approach of Bienstock [3] ; what we will do is guess the 
biggest items for each constraint, rather than the most profitable items. The technique does not 
seem to have an easy analogue for covering problems. 

In detail, previously, we guessed the multiset (7 of 7 most profitable items in the solution. 
Instead, let us guess a /j-tuple {g^,g'^, . . . ,g^) where for each k, 5* is the set of 7 items in the 
solution which have largest coefficients with respect to the ith constraint (breaking ties in each 
constraint in any consistent way). What we need is that any extreme feasible solution with at most 
k fractional values can be rounded to an integral feasible solution at a relative cost factor of at 
most e. Let the original extreme point LP solution be x. We round each fractional value up to 
the closest integer, which causes the solution to become an infeasible one, call it y = [x] . Then, 
to retain feasibility, we go through each of the k constraints, pick the c-smallest set of k items 
from y whose deletion causes the constraint to again become satisfied; and we delete the union of 
these sets from y, obtaining z. Each set has c-cost at most -c{y) since for each constraint i, any 

k elements from g* form an eligible set for deletion, and y C g^ consists of at least 7 items. Thus 
c{z) > c{y) — k-c{y) = (1 — k'^/j)c{x). Taking 7 = k'^/e (compared to the previous k/e), we get 
the desired result. 

5 Discussion 

We believe that the main result is a nice theoretical illustration of techniques (filtering, rounding, 
disjunctive programming). However, it remains to be seen if it could be given useful applications. 
The disjunctive programming trick is definitely senseless sometimes: if we want to write an LP- 
based computer program to (1 + e)-approximately solve multidimensional knapsack instances, it 
is more efficient to consider the LP corresponding to each guess separately (as in [13]) rather 
than solve the gigantic LP obtained by merging them together. Sometimes an LP-relative [19] (or 
Lagrangian-preserving [11, 1]) approximation algorithm can be used as a subroutine in ways that 
a non-LP-relative one could not. However, at least in [11, 19, 1], the analysis relied on LP-relative 
or Lagrangian-preserving analysis of the naive LP, and an arbitrary LP would not have fared as 
well, and the LP we build here seems not to be useful in this way. 

Finding a compact formulation for A;-dimensional covering knapsack with small integrality gap 
and such that the LP does not depend on the objective function is an interesting open problem. 
For example, we are not aware of any polynomial-sized extended LP for 1-dimensional covering 
knapsack with constant integrality gap, in sharp contrast to the packing case. A partial result for 
/c-dimensional covering knapsack is the knapsack-cover LP [6] (see also [2-5, 8, 7] for applications); 



it has integrality gap at most 2A;, and while it is not polynomial size, it can be (l + e)-approximately 
separated [6] and hence (1 + e)-approximately optimized [14, 16] in polynomial time. 

Prom a theoretical perspective, it also seems challenging to find an LP for 2-dimensional (pack- 
ing) knapsack where the size of the LP is a function of 1/e times a polynomial in n, as was done in 
[4] for the 1-dimensional version. 
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